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This paper shows how to compute, for probabilistic hybrid systems, the clock approximation and 
linear phase-portrait approximation that have been proposed for non probabilistic processes by Hen- 
zinger et al. The techniques permit to define a rectangular probabilistic process from a non rectan- 
gular one, hence allowing the model-checking of any class of systems. Clock approximation, which 
applies under some restrictions, aims at replacing a non rectangular variable by a clock variable. 
Linear phase-approximation applies without restriction and yields an approximation that simulates 
the original process. The conditions that we need for probabilistic processes are the same as those 
for the classic case. 
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1 Introduction 

Hybrid processes are a combination of a process that evolves continuously with time and of a discrete 
component. A typical example is a physical system, such as a heating unit, that is controlled by a monitor. 
There are discrete changes of modes, like turning on and off the unit, and there is a continuous evolution - 
the change in temperature. Because of their continuous nature, model-checking hybrid systems can only 
be done for sub-classes of them. Especially, the largest class for which verification is decidable is the 
class of rectangular hybrid automata EIH. Another such class for which verification is decidable is that 
of o-minimal hybrid automata lfT4l . which models hybrid systems whose relevant sets and continuous 
behavior are definable in an o-minimal structure. In the probabilistic case, Sproston proposed methods 
to verify V-PBTL on probabilistic rectangular and o-minimal hybrid processes 1,11,1 . Probabilistic timed 
automata are also a subclass of such processes and have been analyzed extensively ll8l lT2l . 

In order to allow the verification of non-rectangular hybrid automata, two translation/approximation 
methods were proposed by Henzinger et al. (4] : clock-translation and linear phase-portrait approxima- 
tion. The idea behind those methods is to transfer the verification of any hybrid automaton to the one 
of a rectangular hybrid automaton which exhibits the same behaviour or over approximates it. In this 
paper, we show how to apply these methods to probabilistic hybrid processes. We show that both meth- 
ods apply with the same conditions as for the non deterministic case. The technique of approximation is 
based on replacing exact values by lower and upper bounds, after splitting the hybrid automaton for more 
precision in the approximation. Hence, we also show how to split a probabilistic hybrid automaton in 
order to obtain a bisimilar one. Other side contributions of this paper are: a slightly more general, yet a 
simpler definition of probabilistic automata than the one proposed by Sproston [11 J; and the description, 
in the next background section, of the two translation techniques in a simpler way than what can be found 
in Bu. 5], mostly because we take advantage of the fact that the definition of hybrid automata has been 
simplified since then, being presented in terms of functions instead of predicates, and being slightly less 
general than in the original paper (5). 
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2 Transformation methods for hybrid automata 

In this section, we describe the two methods presented by Henzinger et al H that will permit the verifi- 
cation of safety properties on any hybrid system. 

Let X = {xi ,.. . ,Xn} be a finite set of real variables; we write X = {xi , . . . ,i„} where i,- = -;^ is the 
first derivative of x, with respect to time. The set of predicates onXUX is denoted G{XUX). The set of 
valuations a : X — )• M is written M^ or M". A set [/ C R^ is rectangular if there exists a family of (possibly 
unbounded) intervals {Ix)xex with rational endpoints such that [/ = {a G M" | a(x) G 4 for all x G X}. We 
denote by R{X) the set of rectangles over Z. For any set Y, we write ^{Y) (resp. J^fin{Y)) for the power 
set of Y (resp. finite power set of Y). For any variable x, belonging to X or not, we write a [x i— )• r] for the 
valuation that maps x to r G M and agrees with a elsewhere. Conversely, if X' C X, we write a\x' for the 
restriction of a to X'. In the following, we use the notation Set instead of the usual slightly misleading 
one: Reset. 

Definition 1 [1] H = (y,X,Init, Act, Inv, Flow, £", Pre, Set) is a hybrid automaton (HA) if 

• V is a finite set of locations or control modes; 

• X = {xi ,X2, . . . ,x,j} is a set ofn continuous variables; 

• Inv : V — )• =^(M^) defines invariants for the variables in each location. 

• Init : V — )• £P{]R.^) defines initial states and satisfies Init(v) C Inv(v) for all v £V. 

• Act is a finite set of actions, possibly including a silent one, z; 

• Flow : V —7- G{XUX) is afiow evolution condition; 

• £■ C V X Act X V is a finite set of discrete transitions; 

• Pre : E — )• 3^{E^) maps to every discrete transition a set of preconditions; 

• Set : E X M^ — t- ^(M^) describes change in values of variables resulting from taking edges. We 
write S&f{e) := {d(x) | 3a G M^.d G Set(£',a)}. 

H is said to be rectangular if the image o/Inv, Pre and Set are included in R{X) and Flow(v) = 
Ajcez x^Ix where each /v ^ I^ i^ o (possibly unbounded) interval with rational endpoints. 

The semantics of // is a labelled transition system: the set of states is Sh '■= {(v, a) | a G Inv(v)}. 
There are two kinds of transitions between states: flow transitions and discrete transitions. In a flow 
transition, the mode of the automaton is fixed and only the variables' values change over time. More 
formally, there is a flow transition of duration a G M>() between states (v, a) and (v, a'), written (v, a) — ;■ 
(v, a'), if either (1) a = and a = a' or (2) a > and there exists a differentiable function y : [0; a] — )• M" 
with 7: (0;a) -^ W such that /is a solution of Flow(v) with 7(0) = a, Y{a) = a', and /(e) G Inv(v) for 
all £ G (0;a). For discrete transitions, the control mode of the automaton changes instantaneously. We 
write (v, a) — ;■ (v', a'), if there exists e = {v,a,v') G E such that a G Pre{e) and a' G Set(e, a). 

Example 1 Figure^shows the graphical representation of a thermostat [5] that controls the variation 
of temperature in a room through a radiator. The whole system has three modes representing that the 
radiator is either on, off, or down; there is one initial state, where the radiator is on and the temperature 
has value 2. When the radiator is on the temperature increases with respect to the equation x = —x + 5 
whereas it decreases with respect to x= —x when the radiator is off. When the whole system is down, 
no variation of the temperature is modeled. The values of the temperature evolve in the range [1;3]. The 
radiator must switch off when it is on and the temperature reaches 3 units and on when it is off and the 
temperature is 1. Finally, when we try to turn on the radiator, it might turn on or down. 
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Figure 1 : A graphical representation of the thermostat hybrid automaton 
Because we need a notion of weak simulation, we define weak transitions through stuttering. Hence, 

CI X T X a I 

let T be the (usual) silent action. We write 5' ^> i' if there exists a finite sequence s ^ s\ ^ ...^f s^^ s . 
Similarly, we write s ^^ s' ii there exists a finite sequence s -^ s\ ^ S2 -^ ...—)• ijt —^ / such that 
Y,i Oi = o € M>o. Simulation and bisimulation are defined on the underlying infinite transition system 
(and are rather called time bi/simulation in Henzinger et al. Ml/)- 

Definition 2 [4] Let H and H' be two hybrid automata. A relation :<'!Z Sh x Sh' is a simulation ofH 
by H' if every initial state ofH is related by :< to an initial state ofH' and if whenever s :< s', then for 

a a 

each a G Act \ {t} U M>o and each transition s -^ si, there exists a transition s ^* s i such that si :^ s^. 
If ^^^ is also a simulation, then :< is called a bisimulation. If there is a simulation between H and H' 
(resp. a bisimulation), we write H ^H' (resp. H = H'). 

2.1 Clock-translation 

The clock-translation method is based on the substitution of non-rectangular variables by clocks. Let H 
be a non-rectangular hybrid automaton. The substitution of a variable x of // by a clock t,c is possible 
only if, at any time, the value of tx can be determined by the one of x (i.e., x is solvable). 



2.1.1 Preliminaries 

We say that a predicate is simple if it is a positive boolean combination of predicates of the form x ~ c 
where c ^M. and ~G {<,<,=,>,>}■ We say that x is solvable in H if 

• every initial condition, invariant condition, and precondition of H defines simple predicates for 
X and each flow condition of x in Flow(v) has the form (i = /J(x)) A/\, where Px is a simple 
predicate on x; flow evolutions of other variables must not depend on x nor x; 

• the initial- value problem y{t) = fx{y(t))',y{0) =c has a unique, continuous and strictly monotone 
solution ^f.; 



• H is initialised with respect to x. That is, for any transition e £ E, x must either stay unchanged 
in any valuation or get assigned only one value r for all valuations; this will happen if Set'^(e) is a 
singleton with the help of the following notation: we will write 



Sef{e) = {r} C : 



:u{*}, 



where r is either the unique value r € M, in which case we say that x is reset to r by e, or a 
special character, *, which will represent stability in the value of x. If r = * we must also have that 

rv __ jrv' 
J X J X ■ 
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Example 2 The thermostat automaton ofFigureU\is solvable as the flow evolution equation of variable 
X is solvable in all the modes: in mode ON, the differential equation x = —x + 5 with the initial condition 
x(0) = 2 has the function x{t) = —3e^' + 5 where t G M+ as solution. 

Suppose that x £ X is solvable in the hybrid automaton H = (y,X,lnit,Act,lnv,Flow,£',/'re,Set), 
and let c G M be a constant. We say that c is a starting value for a variable a; if c is either: the initial value 
of ;c in some mode v, that is, c = a{x) for a G Init(v); or the unique value of Set^(e) for some edge e G £" 
if this value is not *. Let Dv(x) be the finite set of starting values of x in v. 



Transformation from ;c ~ Z to fjc ~' g^ (l). To simplify the presentation below, we show how predi- 
cates on X are transformed into predicates on t^ |4|. 

Let gc{t) be the unique solution of the initial- value problem 3) (?) =f^{y{t));y{0) =c, where c gM. As 
gc{t) is strictly monotone, there exists at most one? G M+ such that ^^(0 = Z for each Z gM. Let g^^{l) =t 
if gc{t) = I and g^^(Z) = - if gc(?) / Z for all t G M+. Let O := {<,<,=,>,>}. The transformation 
from simple atomic predicates over {x} to simple atomic predicates over {t^} is the function a^ defined 
using r-^G O, It : O ^ O and gt : O ^ O, as follows: 



ac{xr^l) = < 



For each (v, c,) of the hybrid automaton, every predicate ;c ~ Z is replaced by the predicate a^ (;c ~ Z), 
except the invariant predicate which is replaced by ttc, [^ ~ Z) if c,- ~ Z, and hy false otherwise((v, c,) may 
be removed in the latter case). 
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2.1.2 Clock-translation 

We are now ready to define clock-translation. 

Definition 3 l4]IfxGX is solvable in H = ( V,X,Init, Act, Inv, Flow, £, Pre, Set), then the clock-translation 
ofH with respect to x is 

T = (VV , Xj , Inity , Act, Inv^ , Flow 7- , Ej , Prej , Set 7- ) , 

the hybrid system obtained from the following algorithm: 
Step 1: adding the clock t^. 

• Vt '■= Uvev {v} X D^,{x), that is, each mode v ofH is split. Xj := X U {tx}- 

• Init7'(v,c) := {^^[tx 1— )• 0] | a G Init(v) and a(x) = c}. 

• Ej contains two kinds of control switches; for c G Dv{x) and e = (v,a,v') G E 

- if Set^(e) = {r} C M, Ej contains the edge ej '■= {{v,c),a, (v',r)), with Prej{eT) :=Pre{e) and 
Setr(e7-,a) :={d[?vH>0] | d G Set(e,a|x)}. 

- if Set*^(e) = {*}, Ej contains the edge ej := ((v,c),a, (v',c)) with Prerier) '.= Pre(e) and 
Setr(e7-,a) := {d[tx^a{tx)]\deSet{e,a\x)}- 
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Figure 2: The clock-translation of the thermostat automaton 

Step 2: moving to conditions on t^. We view the images o/Init, Inv, Pre, Set and Flow as predicates 
instead of sets of valuations. We replace these predicates over x of the form x ~ r where ~ G {<, <, = 
, > , > } and r £M. in T by predicates over t^ of the form tx ^' gc^i^) '^^ described above. Finally, the 
variable x can be removed from Xj. 

Example 3 The timed automaton of Figure^is obtained by applying the clock-translation on the ther- 
mostat automaton. Each mode v of the automaton is split into |Dv(x)| modes. Since Don{^) = {1;2}, 
we get the modes {ON, 1) and (ON, 2). For these two modes, the differential equations are respec- 
tively "x=—x + 5, where x(0)=2" and "x=—x + 5, where x(0) = l", and then we have the solutions 
"x{t)=—'ie^' + 5 " and "x{t) = —4e^' + 5 " respectively. Next, we substitute the variable x by the clock 
tx in the preconditions, and the invariants. Then, the constraint x < 3 becomes t < ln(2) and t < ln(|) 
respectively. The two automata are bisimilar, by the following theorem. 



Theorem 1 [4] H is bisimilar to its clock-translation T. The relation is given by the graph of the 
projection T] : ^r — )■ Sh, defined as T]((v,c),a):=(v, a'), where a' satisfies a'|x = a|x and s' {x)=gc{;5i{tx)) 
where gc is the solution of the initial-value problem \y{t) = f^{y{t));y{0) = c]. 

As a corollary, H and T satisfy the same properties of usual temporal logics. 

2.2 Linear phase-portrait approximation 

We now present the second method which allows the translation of any hybrid automaton into a rect- 
angular one. The linear phase-portrait approximation method can be applied to any hybrid automaton, 
yielding an approximation of the original process which simulates the original automaton (instead of 
being bisimilar to it, as for clock- translation). This implies that if a safety property is verified on the 
approximation, then it holds in the original system Q. 

The general method is to first split the automaton and then approximate the result. Approximation 
is done by replacing non-rectangular flow equations by lower and upper bounds on the variables, hence 
forgetting the true details of the equations. By splitting more finely, one obtains a better approximation. 



2.2.1 Splitting a hybrid automaton 

Let // be a hybrid automaton with invariant function Inv. A split function is a map 6 that returns to each 
mode V of // a finite open cover {invj , . . . , inv)„ } ^ =^ (M'''- ) of Inv (v) . In splitting, a mode v will be split 
into several modes according to the cover d{v). The fact that U/invJ = Inv(v) makes sure that states are 
preserved whereas the evolution inside mode v is preserved through silent transitions between copies of 
V, which is possible because 6 (v) 's components overlap. 
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L-r=2 




Figure 3: A split of the thermostat 



Definition 4 Let H = (y,X,Init, Act, Inv, Flow, £, Pre, Set) be a hybrid automaton. The split of H by 6 
is the hybrid automaton 6{H) = (ye,X,lnite,Acte,lnve,Flowe,£'e,Pree,Sete) defined as: 

• Ve = {{v,i) \v£V and\<i< \d{v)\} 

• lnite((v,/)) = Init(v) ninv- 

• Acte = ActUJT} 

• lnve(v, /) = invj' 

• Flowe(v, /) = Flow(v) 

• Eq =E\UE2, where El contains the control switch {{v,i) ,a, (V , j)) for each (v,a,v') £E, whereas 
£2 = { ( (v, /) , T, (v, j) ) I (v, /) , (v, 7) G Vg} allows the automaton to transit silently between the dif- 
ferent copies ofv. 

• If ee = {(v,i),a,(v' ,j)) £ El, we set Pree{ee)=Pre{v,a,v') and SetQ{eQ,a) = Set{{v,a,v'),a). If 
e0 = {{v,i),T,{v,j)) ££'2, we set Preg{ee) =M.-^ and Sete{ee, a) = {a}. 

Note that the cover d (v) need not really be open. What is important is that the evolution within any 
mode be preserved, as pointed out in iH. This is the case in the following example, where components 
of the cover are closed and intersect in exactly one point, which is sufficient to allow evolution. 

Example 4 The automaton in Figure pjis a split of the thermostat automaton with function 6 {ON) = 
e{OFF) = {{jc I 1 < X < 2}, {.!c I 2 < ;c < 3}}, and e{DOWN) = {{x\x = 0}}. Note the silent transi- 
tions between states {{0N,i),2), i = 1,2, the latter being duplicates of the original thermostat's state 
{ON, {x I— 7- 2}), that preserve the evolution within mode ON. 



2,2.2 Approximating a hybrid automaton 

An (over) approximation of a HA is obtained by weakening all predicates of its evolution. 

Definitions Let H = (y,X,Init,Act,Inv,Flow,£', Pre, Set) be a HA. Another hybrid automaton A = 
{V,X ,lmtA,Act,Im/A, FlowA,£',PreA,Set^) is a basic approximation ofH if: 

• forallvGV, Inv(v) =^InvA(v), Flow(v) Alnv(v) =^ FIowa (v) A Inv^ (v), Init(v) =^InitA(v); 

• for every discrete transition e £ E, Pre{e) => PreA{e) and Set(e) =^ SetA(e); 

where sets of valuations are viewed as predicates. If there exists a split d on H such that A is a basic 
approximation ofHg then A is a phase -portrait approximation ofH. If the lower and upper bounds of all 
the predicates in A are rational then A is a (rational) linear phase-portrait approximation ofH. 
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Figure 4: A linear phase-portrait approximation of the thermostat 

A straightforward linear phase -portrait approximation is obtained by replacing the invariant in each 
mode V by a product of rational intervals that contains Inv(v) and all other predicates, including the flow 
evolution, by the rational lower and upper bounds implied by the invariant on v. 

Example 5 The automaton ofFigure^is the linear phase-portrait approximation of the thermostat (with 
the same split as in Figure |i]). Every predicate on x is replaced by a predicate that specifies lower and 
upper bounds on it. For example, the approximation ofx in mode (0N,1) yields the set {x \ 3 < x < 4}. 

The following theorem implies that if a safety property is verified for an approximation A, it holds 
also for //Elia. 

Theorem 2 [41 If A is a linear phase-portrait approximation ofH, then A simulates H. If it is just a split 
ofH then A = H. In both cases, the state ((v, /), a) of A is related to (v, a) in H. 

The automaton of Figure |4] simulates the split of the automaton (Figure |3]l, and then, by transitivity 
of simulation, simulates the thermostat hybrid automaton. 

The verification of initialized rectangular hybrid automata has been widely discussed, particularly in 
Q and O and it is proved that their verification is decidable. Hence, a non-rectangular hybrid automata 
can be verified (for satisfaction of safety properties) if an initialized linear phase-portrait approximation 
can be defined from it. 



3 Analysis of probabilistic hybrid automata 

In this section, we show how the two methods presented above can be used for probabilistic hybrid 
automata. Our definition of a probabilistic hybrid automaton (PHA) is close to but slightly more general 
than the one of Sproston 1.11.1 . We also add the definition of finitely branching PHAs. A (discrete) 
probability distribution over a set C is a function /i : C — )• [0, 1] such that Y^cecP^i^) ^ 1^ the support of 
/I is defined as supp(/x) := {c G C | /x(c) > 0}, and it is countable. For [/ C C, we sometimes write 
IJl{U) := Lc-eJ/M(c)- Let Dist(C) be the set of all (discrete) distributions over C. 

Definition 6 A tuple H = (V, X, Init, Act, Inv, Flow, proZ?, {pre^,^a)vev,aeKc\., {pos^,^^)vev,ae Act) is a prob- 
abilistic hybrid automaton (PHA) if V, X Init, Act, Inv and Flow are as in Def ^and 
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Figure 5: A probabilistic version of the thermostat 

• prob : V x Act — )• ^yj„(Dist(y x ^(M^))) encodes probabilistic transitions. IfxG Act, we require 
that every }X G prob{v, t) is concentrated in a unique pair of the form (v, {d}). 

• pre^,^ : prob{v,a) — )• ^(R''^) defines preconditions for distributions from v £V and a G Act. 

• poSy^ : prob{v,a) xV ^ ^(M^) defines postconditions for distributions associated with v £ V 
and a £ Act. 

We say that H is finitely brancliing if for every v £V, a £ Act, jj. G prob{v,a), jJ. is finitely branching, 
that is, supp(/i) is finite and every set post such that (v', post) G supp(/x) is also finite. 

To simplify the notation, we drop the subscripts of pre and pos when there is no ambiguity. 

The semantics of PHAs, is given by probabilistic transition systems. States are defined in the same 
way. As for non-probabilistic hybrid automata, we distinguish two kinds of transitions in PHAs. Flow 
transitions are the same, but discrete transitions are now probabilistic and hence defined from a state 
(v, a) to a distribution. To define transitions, we need some notations on valuations. For d G Mj, a G M^, 
post C Mf , A C M^ and xeX, let 



d[a](x):=r^(") ^^'(")^* and Post[A] :={d[a] | d G post a G A} 
^ ^^ ^ \a{x) if d(A:) = *, post(x):={d(;c) | d G post}. 

Transitions of action a from a state (v, a) in the underlying PTS are as follows. Let /i G prob {v, a), 
a G pre^^^dj.), supp(/x) ={(v,-, post,)}^j. Each combination of d; G post,-, i=l,...,m, such that d,[a] G 
pos{lx,Vi), defines a transition 

(v,a) 



V^\ 



where jX^ is positive on (arrival) states (v,-,d,[a]); the probability that the automaton transits to a state 

(v',a')is 



Ma<''->(v',a') 



^{jU(v,-,post;)|v,=v', d,-[a] = a'} if a'G7?o5(/i,v') 



1=1 







otherwise. 



Example 6 A probabilistic version of the thermostat is shown in Figure |5] Each discrete transition 
is labeled by a probability value and an action. Since there is only one variable, a valuation can be 
represented as a realnumber. Formode OFF, we have prob{OFF ,on) = {/l} where pre{}x) = {1}, such 
that IX {ON, {*}) = 0.9, that is, the temperature is unchanged, and jX (DOWN, {0}) = 0.1. Here, defining 
pos{jX,—) is not necessary since it is encoded in jX. Suppose that in another example one would set 
lx{ON, [1;3]) = 0.9. This would mean that the temperature would end up in the interval [1;3] and that 
the exact value would happen non deterministically. This example would not be finitely branching. 



112 Analysis of Non-Linear Probabilistic Hybrid Systems 

We now discuss how Definition [6] of PHA slightly differs from previous ones lITTl . First note that 
non deterministic transitions in PHA arise in two ways: when the image of prob is not a singleton, and 
from the possible combinations d,- G post,-, i=l,...,m, that we can obtain. Hence, the expression ^zn/tefy 
branching is well chosen because every set post, such that (v, post) G jj., being finite in the underlying 
probabilistic transition system, every state (v, a) will have finitely many distributions /^a associated to 
any action. 

The stai" notation is more general than the reset set of Sproston ifTTl if there is more than one vari- 
able. In the latter, the target of a transition is a distribution over V x ^{W) x ^{X), where the third 
component of a tuple (v, post,X'), called a reset set, represents the set of variables that can change value 
in the transition, with respect to valuations of post. The star notation allows to state, for example, that 
valuation {x,y) will be modified to (x,0), (x,2), (0,^), (1,1) with probability 1 non deterministically. 
The corresponding set would be postg := {(*,0), (*,2), (0,*), (1, 1)}. This is an important feature to 
describe the transitions of the clock-translation (see Section |3.1| ) and is not possible with the reset set 
because there is no uniform reset of any variable in postQ. Note in passing that the star notation avoids 
a third component in the notation and will allow to state very simply the notion of initialised PHA. The 
use of a postcondition function together with the star notation allows to define distributions on complex 
sets, such as: postg n ([0;3] x [1;4]), the distribution being defined on postg and the rectangle being 
the postcondition. We could not transfer the latter into the distributions by defining /i, for example, as 
/i(v', postgn ([0;3] X [1;4])) since postg may contain valuations that assign * to a variable which implies 
that its value depends on the actual state, or valuation for which the distribution jj. will be used. Postcon- 
ditions are necessary for the splitting of PHAs, if one wants to avoid the even more general model that 
consists in defining probabilistic transitions prob from 5// x A to J^fi„{Dist{V x ^(M^))). This is too 
general for practical purposes: indeed, any description of a system must be finite and hence states must 
be described in a parametric (or generic) way. 

The condition on T transitions is to simplify the definition of weak transitions, since it permits to 
write s ^ s'. There may be more than one T transition from a mode v, but for each jj. G prob{v,x), 
there is a unique pair (v', {d}) such that ix{v' , {d}) = 1. Then, following definition of }Xa above, there 
is a T-transition to state (v',d[a]) if and only if d[a] G pos{}jL,v') (that is, /ia(v',d[a]) = 1). Weak flow 



a 



transitions are then defined between states as for hybrid automata, and we write 5 -» ^u if there exists a 

T T X CI 

finite sequence of transitions s ^ s\ -^ S2. ■ ■ ^ Sk ^ jJ.- 

We now define a notion of weak simulation between PHAs. Let ^C 5 x T be a relation between two 
sets S and T. For X C S,we use the notation ^ (X) := {t €^ T \3s £ X.s ^ t}. 

Definition 7 Let Hi, H2 be two probabilistic hybrid automata. A relation ^C 5//, x Sho is a simulation 
if any initial state of Hi is related to an initial state 0///2 and whenever si :< S2, we have: 

• ifsi -^ Hi, for a G S\{t} then S2 -^ IJ.2 and Hi (X) < Hii'^ {X)) for every X ; 

a a 

• if Si -» 5j, for o G M>o, ^2 -^ ^2 ^'^'^ '^i — '*2- 

Then we say that Hi is simulated by H2, written Hi :<H2. If^^^ is also a simulation, it is a bisimulation. 
Equivalently, an equivalence relation is a bisimulation if in the condition above we have }Ji\{X) = pi2{X) 
for each equivalence class X. 

This definition is known to be equivalent to the one using weight functions: see Desharnais et al. |[3l 
for a proof that the inequality between pL\ and pLi above is equivalent to the existence of a network flow 
between them; it is well-known [2J, in turn, that the flow condition is equivalent to the existence of a 
weight function between ixi and 1x2- 
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3.1 Probabilistic clock-translation 

In this section, we prove that clock-translation H, can be applied to a PHA and that it results in a bisim- 
ilar PHA, as expected. The general method is to first compute a non probabilistic hybrid automaton from 
a PHA. Then we apply clock-translation and finally add probabilities in order to obtain a probabilistic 
clock-translation. There is no condition for computing the underlying non probabilistic HA but we need 
a notion of solvability so that we will be able to use the clock-translation method thereafter. A variable x 
of a PHA H is solvable if 

• the two first conditions of solvability for non probabilistic systems are satisfied 

• forevery state5'€ 5//, if i'— 7>/x, and/i(v',post) > 0, then post(x) = {r} for some r€ M*. Moreover, 
if post(A') = {*}, then we must have /_J' = f^ . 

Let Hp = {V,X,lmt,Act,lnv, Flow, prob, {pre), {pos)) be a PHA. The algorithm has three steps: 

Step 1: Define //:=(V,X, Init,A,Inv, Flow, £", Pre, Set) the underlying non probabilistic HA oi Hp as: 

• A := { a^°^* I 3v,v' G V such that jj. G prob(v,a) and /x(v', post) > 0} . 

• £':={(v,a^°^*,v') | /x ^prob{v,a) and /x(v', post) > 0}. Finally, for e := (v,a^°^*,v') G E, we set 
Pre{e) := pre^,^{p.) and Set(e,a) := post[a]. 

Note that solvability ofHp implies that Set^(e) = {r} C R^ and hence H is also solvable. 

Step 2: Since// is solvable, let T = (VV,^7-,Init7',A,Inv7',Flow7,£'r,Pre7-,Set7') be the clock-translation 
of// w.r.t. X. Hence, each transition e = (v,a^°^*,v') of// becomes a transition ej = ((v,c),a^°^*, (v',r)) 
in T , where r = c if post(;c) = {*}, otherwise post(;c) = {r}. 

Step 3: Finally, we build Tp = (yT,XT,lnitT,Act,lm/T,FlowT,Prob, (Pre), (Po^) ), the probabilistic clock- 
translation of Hp from T as follows. Let (v,c) be in Vt, and a in Act. Prob{{v,c),a) contains all distribu- 
tions v^, defined from some jj. G prob{v,a) as follows: 

• for each edge er = ((v,c),a^°^\ (v',r)) such that Set'j.{eT) = {0} (i.e., Sef{e) = {r}), let 

V;i((v',r), post[fv ^ 0]) :=/i(v', post); 

• for each transition er = ((v,c),a^°^*, (v',c)) such that Setj{eT) = Sef{e) = {*}, let 

Vf,{{v,c),post[tjc^*]) :=/x(v',post). 

For both cases, Prf'(v^) ■.= Prej{eT), andPo5(v^, (v',r)) := {a G Mf ^'^'^ | a|x £pos{ix,v')]. 

Example 7 The clock-translation of the probabilistic thermostat automaton is a slight modification of 
the HA of Figure^ all transitions get probability 1 except for on-transitions from {OFF, 3) which have 
probability 0.9 to {ON, 1) and 0.1 to {DOWN, 1). 

We should now prove that the construction yields a valid PHA: this will be a consequence of the 
following theorem. 

Theorem 3 IfTp is the clock-translation ofHp, then Hp and Tp are bisimilar 



1 14 Analysis of Non-Linear Probabilistic Hybrid Systems 

Proof. Let H be the underlying non probabilistic automaton of Hp and T its clock- translation. By 
Theorem [T] H and T are bisimilar through 77 : 5r — )■ 5// which, being a function, returns a unique state 
for any state of T. As Hp and H have the same state space, similarly for T and Tp, T] can be seen as a 
function between states of Tp and Hp. We prove that TJ is a bisimulation between Tp and Hp. 

Let 5 = ((v,c),a) be a state of T^. There are two kinds of transitions to check in the definition of 

simulation. For flow transitions, let 5 -» 5 , where a G M>o. Then since T] is a bisimulation between 

H and T, we obtain T](s) -^ ^7(5''), as wanted. For discrete transitions, we have to prove that for all 
v^ E Prob{{v,c),a) defined from jj. G prob{v,a), for a G Pre{v^) and any combination (d,) from the 

support of v^, we have v^ a (^^U^^)) =Ma| (^) for all U C 5//^. In fact, we need only to prove it for 
f/ equals to some state (v',a') since {(v',a')}U77^^((v',a')) is an equivalence class. 



V, 






(r7-H(v',a')))=I{v;,<^;^((v',r),b)|bU = a',5,(b(^,)) = a'W} 



b,J 



m 
= LLi'^M((^< '''<•), post; 
b,r i=l 



/•(b.H) 



Vi = 

post, 



N 1 Vi = v',, 

^1 post,.W 


r,- = r, 

i = {r 


d,-[a] = 


= b 


1 v; = v',r,- 

POSt;W = 


= {*}: 


c, d,-[a] 
,P{h,c) 


^S 


Vi=v',di[ 

post,(x) = 


a]U = 
{n}, 


b(tv) = 


0> 


v,-=v', d; 

pOSt;(x) = 


[^]\x- 

-{*}, 


= a', 


= a(0 ^ 


v', d,-[a]|x = 


= a', 
3r{*} 


} 





U{v^((v,-,r,-),post- 

m 

= L{v^((v;,'',),post,) 
U{v^((v,-,c),post,-) 

m 

= L{M(v;,post,- 
(=1 

In the third equality, the double sum is reduced to a single one because / determines b and r. D 
Corollary 1 The clock-translation of a solvable PHA is a PHA. 

Proof. We only need to prove that every defined v^ is a distribution, that is, v^ {Sj, ) is 1 . We do 
so by showing that elements of supp(v^) are in bijection with elements of supp(/x). By construction, for 
every d G post such that v^ {s, post) > 0, we have d{tx) = {0} if and only if d(x) = r and d(fv) = * if and 
only if d(x) = *. This implies that d\x = d'\x if and only if d = d', as wanted. Another proof is obtained 
by taking U = Sj^, in the proof of Theorem |3| D 

If all variables of Hp are solvable, then its clock-translation with respect to all its variables will yield 
a probabilistic timed automaton. Knowing that the model-checking of probabilistic timed automata is 
decidable ifTOJ . it implies that the model-checking of solvable PHAs is decidable. 



3.2 Probabilistic linear phase-portrait approximation 

In this section, we show how to apply the linear phase-approximation method to a probabilistic hybrid 
automaton, and that it results in a rectangular hybrid automaton which simulates it. 

Let Hp = {V,X,lmt,Act,lnv,F\ow,prob,{pre),{pos)) be a finitely branching PHA. The method of 
approximation in a probabilistic context follows the same kind of steps as the clock-translation, by going 
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through an underlying non probabilistic hybrid automaton (this approach is also the one of Zhang et 
al. (TT\). However, this translation is simpler, as well as smaller, here as no condition has to be satisfied 
by the non probabilistic automaton. 

Step 1: We define, H = (y,X,Init,A,Inv,Flow,£', Pre, Set), the underlying non probabilistic hybrid 
automaton of Hp as follows: 

• A := {a^ I 3v G V such that jj. G prob{v,a)}. 

• For each /i ^ prob{v,a): E will contain all e = (v,fl:^,v') such that there is some (v',post) S 
supp{lJ.). Pre{e) =pre{lJ.). Set{e, a) = [J post[a]n pos{lJ.,v'). 

(v',post)esupp(/j) 

Step 2: We build the linear phase-portrait approximation of//: 

T = (ye,Xe,Inite,A,Inve,Flowe,£'e,Pree,Sete). 

Step 3: Finally, we build 

r ■.= {Ve,Xe, Inite , Act, Invg , Flowe , Prob, {Pre) , {Pos) ) 

the probabilistic linear phase -portrait approximation of Hp from T as follows. 
Let (v,/) be in Vg, and a in Act. Prob{{v,i),a) contains two kinds of distributions: 

• He defined as follows, for each /i eprob{v,a). For each ee = {{v,i),a^,{v',j)) G Eg and post such 
that (v',post) G supp(/x), we define 

i"e((v',7), post) :=At(v', post), 

and IJ.0 is zero elsewhere. Preconditions and postconditions are 

- Pre{ljLe) :=Pree(ee)ninvJ' and 

- Pos{iJ.e,{v',j)) :=invy nPos{n,v'), 

where inv • := inv'' \ (U^<yinvJ! ) defines a partition of Inv(v'). 

• if a = T, all /i^ defined as 

Hi{{vj),{*r):=\, 

that is, the valuation is unchanged during the silent transition from a copy of v to another. These 
transitions correspond to the edges ((v, /),T, (v,^)) ^Eg. There is no special precondition or post- 
condition, and hence we set Pre{lJ.i) := invj' and Pos{iJ.(, {v' ,j)) := invy . 

Remark 1 The use of postconditions, in presence of the star notation, is crucial in the apparently simple 
definition ofjJ.g above, both to make it correct, and to indeed permit a simple and clean formulation. This 
is on one hand because of the reasons mentioned after Definition [6] On another hand, if we used the 
less general syntax involving reset sets instead of the star notation in distributions, the preconditions of 
jXq would have to take into account the invariant of the arrival state: if a probability is assigned to a 
pair that ends up not being valid because of the splitting of transitions, the probability ends up missing, 
i.e., jJ.g would not sum up to 1: the machinery to overcome this loss of probability would complicate a 

lot the notation. The use of inv j is also crucial in the definition: it makes sure that we do not use, in 
}Xq, a probability value from /I more than once. Indeed, some states get duplicated in the split (if some 
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x = 2 



(ON.l) 
3<i<4 
l<x<2 



(ON.l) 
2<i<3 
2<x<3 



0.9 x=\ 



off 



.v = 3 1 



l<.v<2 



(OFF,!) x=\ "■' foi 
-2<i<-l ^ 



(DOWN,!) 
.v = 
x=0 



(OFF,!) 

-3<i<-2 

2<Ar<3 



Figure 6: A phase-portrait approximation of the probabihstic thermostat 



a belongs to more than one element of G{v)), which has no negative impact in a non probabilistic HA, 
since duplicated transitions define bisimilar systems. However, in the probabilistic case, we must make 
sure that we do not give to both copies the probability value that was meant for one copy: by duplicating 
the value, we would lose the correspondence between copies and the original mode and we could also 
end up with a weight of more than one: this would result in a function that is not a distribution. We chose 
to put the probability on one of the duplicates because silent transitions make sure that the behavior is 
preserved. We could also have chosen to spread the probability to all duplicates uniformly. 



Example 8 A linear phase-portrait approximation of the probabilistic thermostat automaton is obtained 
by slightly modifying the HA of Figure |J] and is illustrated in Figure [^ All transitions get probability 
1 except for on-transitions from {OFF, 1) which have probability 0.9 to {ON, 1) and 0.1 to {DOWN, 1) 
Had we not restricted the postcondition of }Xq{{ON,j), 
{ON, 2) and hence the distribution }Xq £ prob{{OFF , \-),on) would sum up to 1. 



to inv • , it would give probability one to 



We should now prove that the construction yields a valid PHA: this will be a consequence of the 
following theorem. 



Theorem 4 Any linear phase-approximation Hg of a probabilistic PHA Hp simulates it: i.e., Hg :< Hp. 
The split of a PHA is bisimilar to it. 



Proof. Let R be the relation that relates any (v, a) G Sh, to every ((v, /),a), with 1 < / < |0(v)|. Let 
(v, a) G Sh,, a G Act, 1 < / < |0(v)| and/i £prob{v,a). We have to prove that for all a G Pre{lJLe) and any 

combination (dy) from /Iq, we have }X^ {U) < (J-qJ {R{U)) for all U C Sh,. In fact, we show equality. 
This does not give us a bisimulation because of the flow transitions which only satisfy simulation. The 

second equality below relies on the inv^ 's being disjoint and cumulating to inv(v): indeed, for each k, 

there is only one a' G inv^ and all a G inv(v) is in one of those. Consequently the summation over k can 
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be inserted freely. 

l^a (U) = £ £{Ai(v;,post^-) |vy=v', d^[a] = a'}w/ierem=|supp(Ai)| 

{v\a')eUj=i 

|e(v')| m 

= L L L { Ai(vy, post^.) I Vi = v\ dy[a] = a'} 

(v',a')ef/ *:^,,, i=l 
a'Ginvj 

|e(v')l m 
= L L £{Ate((v;,/c),post^.)|v; = v',dj[a] = a'} 

(v',a')ef/ k=\ ,j=\ 
a'einvj 

= I ' i:V^t^((v',^),a') = A^£^-^(^(^)) 

(v',a')eC/ <:=1 ^ 
a'einvJJ 

This completes the proof of the first claim. The second claim follows easily. D 

Example 9 The linear phase-portrait approximation of the probabilistic thermostat automaton ofFig- 
ure^simulates the thermostat automaton of Figure^ In particular, consider the states si = {OFF, 1) 
and S2 = {ON, 1 ) of the original thermostat automaton, such that jX {ON, 1 ) = 0.9 for jx £ prob{OFF, on). 
The states ((0FF,1), 1} and ((0N,1),1) simulate respectively si and S2 since ^e{{ON, 1), 1) = 0.9 for 
He £Prob{{OFF,l),on). 

Corollary 2 Phase-portrait approximation and splitting of finitely branching PHAs are PHAs. 

Proof. We only need to prove that every defined /i^a is a distribution, that is, the total probability 
out of ;Uea is 1. This is guaranteed by /ig being a distribution and by taking U := Sg in the proof of 
Theorem [4I one obtains that IXa{SH) = l-iea{Se). Since the distribution has nothing to do with the flow 
evolution, it is the same argument for both cases of approximation and splitting. D 

Since an approximation of a probabilistic hybrid automaton is a rectangular PHA, its model-checking 
is decidable [111. Therefore, by taking the right approximation to it, any probabilistic hybrid automaton 
can be verified. 



4 Conclusion 

In this paper, we proved that the two methods of Henzinger et al. [4], clock-translation and linear phase- 
portrait approximation, can also be applied in the probabilistic context to verify non-rectangular PHAs. 
The adaptation of the methods to PHAs were facilitated by a modification of the syntax over PHAs: a 
star notation to represent stability of a variable after a transition and postcondition functions. The advan- 
tage of adapting the methods instead of defining them from scratch is mainly that proving bi/simulation 
had only to be checked for discrete transitions. The correctness of the constructions is ensured by 
bi/simulation relations. 

The first method, when it is applicable, results in a probabilistic timed automaton which satisfies 
exactly the same properties as the original PHA. The inconvenience of this method is that it requires, as 
in the non-probabilistic case, that the non-rectangular variables be solvable: all the equations induced by 
the flow evolution have solutions in M. 
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For linear phase-portrait approximation, there is no restriction on the PHA, and its application results 
in a rectangular PHA that simulates the original one. When a safety property is satisfied by the rectangu- 
lar approximation, we can assert that the original PHA satisfies the same property. However, in the case 
when a safety property is not satisfied by the approximation, more splits should be done; this could be 
costly in time depending on the property and the size of the PHA. 

Side contributions of this paper are also: new additions to the definition of PHA that make them more 
general; a splitting construction on PHAs that results in a bisimilar PHA; a simpler description of the two 
techniques than what can be found in the original papers Hm. About the additions to the definition of 
PHAs, it is interesting to note that the star notation and the postcondition function on distributions permit 
to represent constraints on variables in terms of set of valuations instead of predicates. When working 
with distributions, set of valuations are more natural than predicates. 

Let us discuss how our approximation relates to the construction of a recent paper by Zhang et al |[T3]| . 
which also defines approximations for probabilistic hybrid automata. That paper also gives a method to 
over approximate the original automaton. It is clear that the latter is less abstract than the former since 
Zhang et al. define a. finite approximation. As in the construction of Henzinger et al. [4|, they start from a 
cover of the state space and abstract according to it. The difference is that the cover is over states instead 
of over the variables' space of values. More importantly, whereas we use the cover to "linearize" each 
piece that the cover defines, they group together all these states into one single state. The result is a finite 
probabilistic transition system, whereas we obtain a PHA. They prove, as we do, that their abstraction 
simulates the original PHA. However, their approximation is more abstract, which has the advantage of 
being smaller but of course with less information. 

Future work includes the implementations of the technique into a probabilistic model checker and 
taking advantage of other approximation techniques that have been developed for probabilistic systems 
in order to widen the class of PHAs for which model-checking is supported. 
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